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1.  Introduction 


The  purpose  of  this  research  was  to  determine  if  networked  battle  command  provides  command 
teams  with  a  shared  understanding  (or  shared  mental  model)  of  the  situation  to  enable  collabora¬ 
tive  planning  and  execution.  In  order  for  the  U.S.  Army  future  force  to  succeed,  that  force  will 
have  to  “see  first,  understand  first,  and  act  first”  (Department  of  the  Army,  2003).  Compared  to 
the  current  force,  the  future  force  systems  will  need  to  be  much  lighter  and  smaller  to  increase 
their  ability  to  deploy.  Thus,  in  order  to  ensure  Soldier  survivability,  the  future  force  must  use 
sensors,  including  unmanned  aerial  vehicles  and  unmanned  ground  vehicles  to  see  the  enemy  at  a 
distance  (see  first).  The  future  force  must  then  use  networked  battle  command  and  a  common 
interface  to  provide  the  common  operational  picture  (COP)  to  all  platforms  (understand  first). 
Finally,  the  future  force  must  use  their  agility  and  precision,  long  range  fires  in  order  to  engage 
the  enemy  at  a  time  and  place  of  the  unit’s  choosing  (act  first). 

The  future  combat  system  (FCS)  operational  and  organizational  (O&O)  plan  emphasizes 
collaborative  planning  and  execution,  particularly  among  commanders,  in  order  to  see  first, 
understand  first,  and  act  first  (U.S.  Army  Training  and  Doctrine  Command,  2002).  This 
collaborative  planning  and  execution  is  enabled  by  a  shared  mental  model.  That  is,  Soldiers 
must  have  a  common  understanding  about  those  areas  for  which  the  plan  requires  information  so 
that  assets  which  enable  the  force  to  see  first  can  be  properly  deployed.  The  unit  as  a  whole  must 
have  a  common  understanding  about  the  current  situation  and  the  potential  problems  in  each 
other’s  areas  of  responsibility,  so  that  the  unit  understands  first.  This  common  understanding 
enables  the  unit  to  act  first,  not  as  uncoordinated  individuals  or  sub-units,  but  as  a  coherent 
whole. 

Research  by  Stout,  Cannon-Bowers,  Salas,  and  Milanovich  (1999)  using  a  flight  simulation  task, 
showed  that  teams  with  more  similar  shared  mental  models  did  better  planning,  “pushed”  more 
information  (i.e.,  provided  infonnation  without  it  having  to  be  requested),  and  made  fewer  errors. 
Mathieu,  Goodwin,  Heffner,  Salas,  and  Cannon-Bowers  (2000)  showed  that  team  mental  model 
congruence  was  positively  related  to  team  processes  and  team  performance  in  a  flight  simulation 
task.  Marks,  Zaccaro,  and  Mathieu  (2000),  in  a  simulated  armor  platoon  task,  found  that  team 
mental  model  similarity  was  positively  related  to  team  processes  and  in  novel  situations,  to  team 
performance  as  well. 

It  is  logical  that  command  teams  with  better  shared  mental  models  would  have  better  situational 
understanding  (SU)  and  be  more  effective.  It  is  likely  that  those  who  better  understand  how  each 
team  member  sees  the  situation  will  be  able  to  focus  their  efforts  on  attaining  more  relevant 
information  and  thus  increase  their  understanding.  Along  with  increasing  their  SU,  it  is  likely 
that  they  could  also  increase  their  effectiveness  by  developing  more  relevant  plans  or  taking 
more  relevant  actions. 
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The  purpose  of  this  research  was  to  examine  commander  teams’  mental  models,  perceived 
workload,  SU,  and  effectiveness  at  unit  of  employment  (UE),  unit  of  action  (UA),  and  combined 
arms  battalion  (CAB)  levels.  If  the  new,  network-centric  tools  and  organization  of  the  future 
force  are  to  succeed,  they  will  have  to  facilitate  the  development  of  shared  mental  models  for 
command  teams  at  various  levels  by  providing  common  information,  which  aids  in  developing 
SU  and  reduces  workload.  SU  for  this  experiment  is  defined  as  comprehension  of  the  friendly 
and  enemy  situation  and  projecting  what  actions  need  to  be  taken  as  a  result  of  the  situation. 


2.  Method 


2,1  Experimental  Overview 

This  experiment  was  performed  at  the  Unit  of  Action  Maneuver  Battle  Lab  (UAMBL),  at  Fort 
Knox,  Kentucky,  from  June  1  through  26,  2004.  The  experiment  simulated  battle  command  in  a 
UA  and  used  future  organization,  doctrine,  and  a  surrogate  of  the  future  battle  command 
interface  (maneuver  command  and  control  or  MC2).  The  simulation  consisted  of  13  days  of 
individual,  cell  (e.g.,  first  UA  mobile  command  group  1;  a  brigade-like  tactical  operations 
center-vehicle  containing  six  key  personnel),  and  unit  training  and  pilot  tests,  and  six  days  of 
experimental  runs. 

The  experimental  unit  organization  consisted  of  one  UE  (division-sized  element),  with  a  partial 
staff,  and  several  subordinate  UAs  (brigade-sized  elements  consisting  of  three  maneuver  UAs,  a 
strike  or  aviation  UA,  and  a  fires  UA).  Only  one  UA  had  a  substantial  staff,  with  all  staff 
organizations  represented,  and  six  subordinate  battalion-level  organizations;  three  CABs,  a  non- 
line  of  sight  (NLOS)  battalion,  a  forward  support  battalion,  and  an  aviation  battalion.  Most  of 
these  battalions  were  represented  by  a  small  staff  (two  CABs  had  only  a  commander),  but  one 
subordinate  CAB  had  a  substantial  staff  and  six  subordinate  company-level  organizations  (two 
mounted  combat  system  companies,  two  infantry  companies,  a  reconnaissance,  surveillance  and 
target  acquisition  [RSTA]  company,  and  an  NLOS  mortar  battery).  Two  companies  had 
subordinate  platoons.  One  of  these  two  companies  had  subordinate  platoon  leaders  and  platoon 
sergeants  only.  The  remaining  company  had  subordinate  platoon  leaders,  platoon  sergeants,  and 
three  squad  leaders  in  each  platoon.  The  rest  of  the  companies  were  represented  by  a  company 
commander  only.  This  design  is  referred  to  as  a  “slice”  design,  where  at  least  one  level  of  the 
organization  is  fully  shown  from  lowest  to  highest — in  this  case,  platoon  through  UE  (see 
figure  1). 

The  rest  of  the  UAs  were  played  constructively,  represented  at  most  by  a  small  staff  consisting  of 
a  few  “live”  participants,  with  research  assistants  assigned  to  control  subordinate  battalions  (the 
two  maneuver  UAs  were  played  by  only  a  UA  commander). 
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Figure  1.  Organization  slice  design. 


2.2  Participants 

Participants  were  organized  into  command  teams  in  the  following  manner: 

UE  command  team  -  UE  commander;  first,  second,  and  third  UA  commanders,  fires  UA 
commander,  aviation  UA  commander. 

UA-level  commanders  -  first  UA  commander;  first,  second,  and  third  CAB  commanders, 
NLOS  battalion  commander,  forward  support  battalion  commander,  aviation  squadron 
commander. 
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CAB-level  commanders  -  second  CAB  commander;  A,  B,  C,  D,  company  commanders, 
RSTA  company  commander,  mortar  battery  commander. 

2.3  Scenario1 

2.3.1  Mission  and  Array 

The  mission  of  “blue”  ground  forces  was  to  attack  in  zone  to  defeat  enemy  forces  in  a  middle 
eastern  country  to  facilitate  the  reinstatement  of  the  legitimate  government.  Blue  forces 
consisted  of  a  joint  task  force  with  a  combined  forces  land  component  command,  in  which  the 
Army  component  consists  of  a  UEy,  with  a  Stryker  division  (SDIV)  and  a  UEx.  The  UEx  was 
composed  of  three  maneuver  UAs.  The  UA  force  design  depicted  in  this  analysis  is  based  on  the 
increment  1  resource-constrained  threshold  (2012)  design  depicted  in  the  30  June  2003  change  2 
to  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Pamphlet  525-3-90  (2002).  The 
UEx  and  UEy  designs  were  developed  after  discussions  and  coordination  with  the  combined 
anns  center  and  TRADOC. 

2.3.2  Intent  and  Concept  of  Operations 

The  intent  of  blue  operations  was  for  the  three  maneuver  UAs  to  execute  a  river  crossing  and 
conduct  a  forward  passage  of  lines  to  use  UAs  in  turn  to  destroy  threat  defense  belts.  This 
penetration  would  enable  the  passage  of  the  SDIV  to  eliminate  threat  forces  and  regain  control  of 
the  capital  city,  enabling  the  coalition  to  restore  the  legitimate  government.  The  concept  of 
operations  called  for  UA1  to  destroy  the  first  belt  of  threat  forces  to  facilitate  the  forward 
advance  of  the  other  UAs.  UA2  would  engage  and  destroy  second  belt  threat  forces  to  the  front 
and  left  flank  to  facilitate  the  forward  passage  of  UA3  and  to  establish  the  UEx  line  of 
communication.  UA3  would  then  exploit  the  penetration  and  attack  remaining  threat  forces  west 
of  the  capital  city,  the  enemy  center  of  gravity.  The  intent  is  to  develop  the  situation  out  of 
contact,  conduct  stand-off  engagements  to  set  the  conditions  for  close  assault,  and  rapidly 
transition  to  ensuing  operations. 

Table  1  shows  the  experimental  schedule.  The  pilot  test  was  an  attack  in  open  terrain  to  isolate 
the  city.  Trial  1  was  also  an  attack  in  open  terrain,  with  the  network  functioning  at  100%  (non- 
degraded,  i.e.,  no  interference  with  digital  or  voice  communications  over  the  network).  Trial  2 
was  a  repeat  of  the  attack  in  open  terrain  but  with  the  network  functioning  at  around  75%.  The 
experimental  runs  in  the  second  week  (2 1  to  25  June)  represent  a  mixture  of  full  and  reduced 
network  functionalities,  so  they  cannot  be  characterized  as  trial  1  or  2. 


'Taken  from  U.S.  Army  TRADOC  Analysis  Center  report  (June  2004)  Future  Combat  Systems  analysis  of  key 
performance  parameter  2,  Networked  Battle  Command. 
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Table  1.  Missions  by  days  (June  04) 


Monday 

Tuesday 

Wednesday 

Thursday 

Friday 

14 

15 

16 

17 

18 

Pilot  Test  1 

Pilot  Test  2 

Pilot  Test  3 

Trial  1 

Trial  2 

Attack  in  open 

Attack  in  open 

Tactics,  techniques, 

Attack  in  open 

Attack  in  open 

terrain  to  isolate 
city 

terrain  to  isolate 
city 

procedures  development 

100%  of  network 

terrain  to  isolate 
city 

terrain 

100%  of  network 
function 

100%  of 
network 
function 

function 

100%  network 
function 

75%  network 
function 

21 

22 

23 

24 

25 

Day  1 

Day  2 

Day  3 

Day  4 

Day  5 

Set  Conditions 

Maneuver 

Seize  key  terrain 

Isolate  Center  of 
Gravity 

Attack  Center  of 
Gravity 

Mixed  Network 

Mixed  Network 

Mixed  Network 

Mixed  Network 

Mixed  Network 

Functioning 

Functioning 

Functioning 

Functioning 

Functioning 

2.4  Experimental  Limitations 

Workload,  SU,  and  shared  mental  models  assessment  was  constrained  because  of  the  following 
limitations  of  the  experiment: 

a.  The  experiment  was  not  strictly  controlled  (free  play  versus  scripted  scenario),  so  that 
objective  measures  of  SU  and  shared  mental  model  could  not  be  collected. 

b.  Because  of  the  free  play  allowed  in  the  scenario,  there  was  no  pre-determined  “ground 
truth”  at  specific  times  to  assess  SU  and  shared  mental  models  against. 

c.  Command  teams  were  not  well  trained  and  were  ad  hoc  in  nature.  Training  for  the 
experiment  was  limited.  The  command  teams  consisted  of  participants  assigned  to  certain 
positions  for  the  duration  of  the  experiment  with  no  previous  experience  working  together. 
This  limitation  presents  a  major  problem  in  the  interpretation  of  the  results,  since  it  is 
difficult  to  determine  whether  shortcomings  in  the  shared  mental  model  should  be 
attributed  to  lack  of  training  or  to  other  factors. 

2.5  Data  Collection  and  Analysis 

Surveys  for  data  collection  were  administered  before  the  lunch  break  and  (or)  at  the  end  of  the 
day.  They  were  administered  on  the  same  computer  that  participants  used  to  perform  their  tasks 
during  the  trial.  Surveys  were  given  at  these  times  to  collect  data  while  they  were  still  fresh  in 
the  participant’s  experience  but  did  not  interfere  with  the  experiment. 

Workload  consisted  of  measures  shown  in  table  2.  These  workload  measures  were  assessed  on  a 
seven-point  scale  from  low  (1)  to  high  (7)  and  divided  by  seven  so  that  they  are  on  the  same  0  to 
1  scale  as  the  shared  mental  model  data.  Thus,  an  average  score  of  5  (of  7)  on  a  scale  (e.g., 
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frustration)  would  result  in  a  figure  of  71%  frustration.  This  is  the  percentage  reported  on  the  y- 
axis  in  the  figures  in  this  report.  This  scale  normalization  allowed  easy  comparison  of  workload 
and  mental  model  congruence.  An  overall  measure  of  workload  was  also  computed  by  the 
addition  of  the  four  measures  of  workload  for  each  participant.  We  then  divided  by  four  and 
again  divided  by  seven,  so  that  the  measure  of  workload  is  on  a  0  to  1  scale  comparable  to  the 
shared  mental  model  data. 


Table  2.  Workload  measures 


Workload  Measure 

Definition 

Frustration 

Degree  to  which  participants  were  prevented  from  achieving  their  goals 

Mental  Demand 

How  hard  participants  perceived  they  had  to  think 

Temporal  Demand 

How  much  time  pressure  participants  perceived 

Effectiveness 

How  proficient  participants  perceived  they  were  at  performing  their  tasks 

Overall  Workload 

Averaged  sum  of  above  four  workload  scales 

Perceived  SU  was  measured  on  a  seven-point  scale,  from  low  (1)  to  high  (7).  We  also  divided 
the  score  by  seven  so  that  it  is  on  a  0  to  1  scale  similar  to  the  shared  mental  model  data.  We  used 
perceived  SU  versus  a  more  objective  measure,  since  the  free  play  nature  of  the  experiment  made 
it  difficult  to  assess  “ground  truth.” 

Shared  mental  model  assessment  was  derived  from  commander  and  team  mental  model 
congruence.  Commander  and  team  congruence  were  computed  based  on  answers  to  two 
questions  in  a  daily  survey.  These  questions  were 

What  is  the  most  significant  threat  to  current  operational  success? 

What  is  the  second  most  significant  threat  to  current  operational  success? 

We  used  a  technique  developed  by  Lieutenant  Colonel  John  Graham2  to  measure  mental  model 
congruence.  Participants  were  asked  to  indicate  the  most  significant  and  second  most  significant 
threat  to  current  operations,  choosing  from  a  list  of  22  possible  threats.  These  threats  consisted 
of  eight  battlefield  functional  areas  each  for  friendly  and  enemy  forces,  plus  six  environmental 
factors.  The  22  threats  are  shown  in  tables  3  and  4.  The  degree  to  which  any  two  participants 
agreed  on  these  ratings  was  assessed  on  a  0-to-l  scale,  with  0  representing  no  congruence  and  1 
representing  complete  congruence. 

Commander  congruence  was  calculated  as  an  average  of  the  subordinate  commanders’ 
congruence  with  the  superior  commander’s  mental  model  (e.g.,  average  congruence  of  the  first, 
second,  and  third  UA  commanders,  fires  UA  commander,  and  aviation  UA  commander  with  the 
UE  commander’s  mental  model).  Team  congruence  was  calculated  as  the  average  congruence 
among  the  subordinate  commander  team  members.  It  does  not  include  the  superior  commander 
so  that  it  is  completely  independent  from  the  commander  mental  model. 
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Table  3.  Current  threats  for  battlefield 
functional  area 


Battlefield  Functional  Areas* 
Command  and  control 
Sustainment 

Information _ 

Communication _ 

Maneuver 

Fire  Support _ 

Air  Defense _ 

Aviation _ 

*both  friendly  and  enemy 


Table  4.  Current  threats  for 
environment 

Sensors _ 

Terrestrial  weather 

Space  weather _ 

Terrain _ 

Time _ 

Civilian  population _ 


Both  commander  and  team  congruence  were  reported  on  a  0-to-l  scale,  where  0  represents  no 
congruence  between  mental  models  and  1  represents  complete  mental  model  congruence. 

Data  analyses  consisted  of  descriptive  statistics  only.  The  method  used  to  compute  means  for 
workload,  SU,  and  mental  model  congruence  by  the  three  groups  of  commanders  were  described 
above.  Because  of  the  small  number  of  participants,  no  inferential  statistics  were  used. 


3.  Results 


3.1  Situational  Understanding  and  Workload 

For  UE-  and  UA-level  commanders,  SU  and  effectiveness  workload  ratings  are  relatively  high 
(in  the  70%  range)  and  generally  show  a  slight  increase  over  time,  suggesting  increased 
perceived  proficiency  (see  figure  2).  Other  workload  measures  (frustration,  mental,  and 
temporal  demand)  start  in  the  50%  to  60%  range  and  generally  decline  to  the  40%  to  50%  range 
over  time,  also  suggesting  increased  proficiency  resulting  in  reduced  perceived  workload.  This 
may  suggest  a  learning  curve  over  the  course  of  the  experiment.  These  data  suggest  that  the 
battle  command  tools  were  effective  in  providing  a  moderate  workload  and  good  SU  for  the  UE 
command  team. 

Similarly,  figure  3  for  UA  and  battalion-level  commanders  shows  that  SU  and  effectiveness  were 
relatively  high  (in  the  70%  range),  with  a  slight  increase  toward  the  end  of  the  exercise,  while 
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other  measures  of  workload  were  lower  (overall  workload  in  the  60%  to  50%  range)  and  showed 
a  slight  decline  over  time.  Again,  improvement  over  the  course  of  the  experiment  suggested  that 
a  learning  curve  is  one  possible  interpretation.  Although  no  formal  surveys  were  administered, 
the  subject  matter  expert  (SME)  observing  the  UE  command  cell  suggested  reasons  for  the 
relatively  low  workload  ratings.  One  reason  may  have  been  an  internal  comparison  with  the 
workload  induced  by  this  8-  to  10-hour-a-day  experiment  to  the  workload  at  the  National 
Training  Center  (NTC)  or  the  Battle  Command  Training  Program  exercises  running  18+  hours  a 
day.  In  addition,  the  factors  of  vehicle  motion,  noise,  heat,  and  the  more  hectic  pace  at  the  NTC 
may  have  made  the  workload  of  this  experiment  pale  in  comparison.  Generally,  these  data 
suggest  that  the  battle  command  tools,  including  the  MC2  interface,  provided  a  moderate 
workload  and  contribute  to  good  SU.  This  conclusion  is  entirely  based  on  subjective  data.  The 
more  objective  techniques  such  as  Endsley’s  (2000)  situational  awareness  global  assessment 
technique  (SAGAT)  could  not  be  used  because  of  the  free  play  nature  of  the  experiment. 


UE  and  UACdrs 


AM  AM  PM  AM  PM  PM  AM  PM 

Dates 


— Frustration 

— < ■—  Mental 

Temporal 

Effectiveness 

— *—  Overall  Workload 

-•-SU 

Figure  2.  Workload  and  SU  for  UE  command  team. 
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UA  and  Bn  Cdrs 


AM  AM  PM  AM  PM  PM  AM  PM 

Dates 


— Frustration 

— ■—  Mental 

Temporal 

Effectiveness 

x  Overall  Workload 

SU 

Figure  3.  Workload  and  SU  for  UA  command  team. 

For  battalion-  and  company-level  commanders  (figure  4),  SU  was  quite  high  (around  90%)  and 
relatively  steady  over  time.  Effectiveness  declined  from  initially  high  levels  (in  the  80%  range) 
to  70%  about  mid-exercise,  and  then  increased  toward  the  end  of  the  exercise.  One  possible 
reason  for  the  dip  in  effectiveness  may  be  model-induced  problems  in  fuel  and  ammunition 
shortages.  This  caused  the  commanders  to  spend  considerable  time  on  re-supply  issues.  The 
effectiveness  improvement  shown  later  may  have  been  attributable  to  white  cell  (the  cell  that 
serves  as  higher  headquarters  and  also  directs  the  experiment)  “work-arounds”  in  re-supply  to 
compensate  for  perceived  model  inaccuracies  in  usage  rates.  Another  possible  reason  for  this  dip 
in  perceived  effectiveness  could  be  the  slow  rate  of  revising  the  COP  during  this  period.  The 
perceived  SU  also  dipped  slightly  during  this  period,  supporting  this  latter  interpretation.  Other 
workload  measures  were  relatively  steady  over  the  exercise  (overall  workload  around  60%). 

Thus  for  the  command  team  involved  in  the  lowest  levels  of  the  city  fight  (24  to  25  June), 
workload  did  not  decline  toward  the  end  of  the  exercise.  Even  so,  these  data  taken  as  a  whole 
suggest  that  the  battle  command  tools  enabled  command  teams  to  maintain  moderate  workload 
(50%  to  60%  range)  and  good  SU  (80%  to  90%  range). 
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CAB  and  Co  Cdrs 


18JunAM  23JunAM  24  Jun  PM  25JunAM  25  Jun  PM 

Date 


Workload  -»-SU  CDRMM  TM  MM 


Figure  4.  Workload  and  SU  for  battalion  command  team. 


3.2  Mental  Model  Data 
3.2.1  UE  Command  Team 

As  shown  in  figure  5,  the  UE  command  team  mental  model  varied  significantly  in  the  first  few 
days  of  the  experiment,  then  remained  relatively  stable  for  the  rest  of  the  experiment,  with  a 
slight  increase  on  the  last  afternoon.  Overall  though,  mental  model  congruence  was  low  (in  the 
20%  and  30%  range)  across  the  staff.  No  data  were  available  for  the  UE  command  team  for 
18  June  p.m.  (afternoon)  because  the  commander  did  not  complete  the  mental  model. 

The  commander’s  mental  model  is  more  congruent  than  the  team  mental  model  during  most  of 
the  experiment,  which  suggests  that  subordinate  commanders’  understanding  of  their 
commander’s  mental  model  is  more  congruent  than  their  understanding  of  each  other’s.  This 
suggests  that  the  commander  kept  subordinates  informed  of  what  he  thought  were  important 
threats  to  the  mission  better  than  subordinate  commanders  kept  each  other  informed  of  their 
perceived  threats  to  the  mission. 


10 


UE  Command  Team 


Dates 


UE  CDR  MM  UE  TM  MM 


Figure  5.  UE  command  team  mental  model  data. 

SMEs  at  the  UE  stated  that  the  fall  in  the  commander  mental  model  and  peak  in  the  team  mental 
model  on  the  afternoon  of  2 1  June  was  attributable  to  the  same  circumstance.  Near  the  end  of 
that  day,  before  the  survey,  the  UE  commander  was  discussing  what  to  do  about  the  objective  on 
an  escarpment  and  the  actions  that  UA  commanders  needed  to  take  to  achieve  that  objective. 
However,  he  suddenly  changed  his  focus  from  the  tactical  level  to  setting  conditions  and  shaping 
the  battle  two  days  away.  This  appeared  to  escape  the  notice  of  the  UA  commanders,  who 
remained  focused  on  the  near-term  fight.  Thus,  although  the  UA  commanders  had  a  reasonably 
congruent  mental  model  of  the  situation,  it  did  not  synchronize  with  the  UE  commander’s  mental 
model  at  that  time. 

The  slight  rise  in  both  mental  models  on  the  last  day  could  be  a  result  of  no  future  planning  being 
necessary,  since  the  experiment  was  ending. 

However,  compared  to  self  ratings  of  workload  and  SU,  the  mental  model  congruence  data  for 
UE  and  UA  commanders  are  quite  low  (generally  in  the  20%  to  30%  range),  never  exceeding 
55%.  This  suggests  that  command  teams  lacked  a  shared  understanding  of  doctrine,  tactics, 
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techniques  and  procedures  (TTP),  and  organization,  perhaps  because  of  inadequate  training  or 
the  battle  command  tools  were  not  sufficient  for  commanders  to  develop  congruent  mental 
models. 

3.2.2  UA  Command  Team 

As  shown  in  figure  6,  the  UA  team’s  mental  model  remained  relatively  steady  over  time, 
suggesting  that  although  they  had  a  low  knowledge  base  of  understanding  each  other’s  perceived 
significant  threats,  this  knowledge  base  remained  fairly  consistent  over  time,  while  their 
knowledge  base  of  the  commander’s  perceived  threat  varied  significantly  in  several  instances. 
This  suggests  that  the  commander  may  have  had  some  difficulties  keeping  his  subordinates 
informed  of  what  he  thought  were  important  threats  to  the  mission,  perhaps  because  of  the  lack 
of  established  TTPs. 

The  data  show  a  general  decline  in  the  commander’s  mental  model  through  the  end  of  June  23 
(seize  key  terrain),  with  a  sharp  rise  in  congruence  on  June  24  and  25  (isolate  center  of  gravity). 
Once  this  fight  was  finished,  there  was  again  a  decline  in  mental  congruence. 

A  possible  explanation  for  the  decline  and  low  point  on  June  23  was  that  the  UA  commander 
spent  most  of  the  time  out  of  his  “vehicle”  working  on  other  duties  (or  conducting  “face  to  face” 
meetings  with  other  commanders  and  staff),  and  the  deputy  commander  was  in  charge.  The 
decline  toward  the  end  of  the  exercise  may  reflect  the  difficulty  of  the  urban  fight.  Another 
interpretation  for  the  decline  is  that  battle  “hand-off,”  that  is,  transfer,  between  the  commander 
and  deputy  commander  may  need  to  be  improved.  Or  it  may  just  be  an  “end  of  test”  artifact. 

The  CAB  command  team  mental  model  shown  in  figure  7  shows  a  general  decline  in  mental 
model  congruence  throughout  the  test,  except  for  a  peak  (for  commander  mental  model)  on 
June  21  (set  conditions).  The  generally  low  (in  the  20%  to  50%  range)  and  declining  congruence 
of  mental  models  may  have  resulted  from  (a)  infantry  company  commanders  being  at  Fort 
Benning,  Georgia,  and  not  being  infonned  about  the  end-of-day  after-action  reviews  (AARs)  and 
(b)  coaching  from  the  battalion  commander  (congruence  for  these  commanders  was  lower, 
particularly  earlier  in  the  experiment).  The  AARs  were  face  to  face  and  thus,  the  remote  site  at 
Benning  did  not  participate.  This  suggests  that  the  remote  units  need  to  be  included  in  the  AARs 
and  coaching  from  the  battalion  commander,  so  that  we  train  as  we  are  going  to  fight,  as 
specified  by  doctrine. 

The  severe  dip  in  commander  mental  model  congruence  on  June  18  p.m.  (also  the  UE  mental 
model  is  declining  at  this  point)  may  have  been  attributable  to  a  perceived  threat  from  the  west  of 
phase  line  lion.  The  UA  staff  thought  they  detected  a  large  mechanized  force  preparing  to  attack 
into  the  flank  of  the  2nd  CAB.  However,  later  analysis  revealed  this  to  be  a  much  smaller  enemy 
force.  The  CAB  commander  then  perceived  the  smaller  force  as  a  target  of  opportunity,  given 
the  CAB’s  ability  to  see  and  strike  from  a  distance.  However,  the  company-level  commanders, 
who  had  less  understanding  of  the  future  forces  capabilities,  still  saw  the  enemy  as  a  substantial 


12 


threat  since  they  did  not  understand  the  capability  to  destroy  the  enemy  at  a  distance.  This 
suggests  that  lack  of  understanding  of  doctrine  (resulting  from  a  lack  of  training)  can  hinder 
development  of  congruent  situational  mental  models. 


UA  Command  Team 


AM  PM  PM  AM  AM  PM  AM  PM 

Dates 


UA  CDR  MM  UA  TM  MM 


Figure  6.  UA  command  team  mental  model  data. 

The  peak  in  commander  mental  model  congruence  on  the  afternoon  of  June  2 1  could  be  related 
to  the  2nd  CAB  remaining  in  the  tactical  assembly  area  all  day,  while  the  1st  and  3rd  CABs 
established  conditions  favorable  to  the  2nd  CAB’s  assault.  The  2nd  CAB  was  involved  in 
providing  security  and  integrating  aviation  assets  during  this  period.  Since  the  staff  had 
relatively  little  to  do  and  their  situation  in  the  assembly  area  did  not  change,  maintaining  a 
congruent  mental  model  was  easier. 

Also,  there  was  a  considerable  difference  in  experience  and  therefore  knowledge  of  the  O&O 
concept  and  opposing  forces  capabilities  between  the  CAB  and  company  commanders.  This  lack 
of  understanding  could  have  exacerbated  differences  in  mental  models  over  time  in  the 
experiment.  These  data  point  to  the  need  to  understand  the  future  force  doctrine  in  order  to  have 


13 


congruent  mental  models.  This  finding  suggests  that  better  training  of  future  force  doctrine  is 
needed  to  improve  understanding,  which  then  leads  to  the  development  of  more  congruent 
mental  models. 


CAB  Command  Team 


Dates 

CAB  CDR  MM  CAB  TM  MM 


Figure  7.  CAB  command  team  mental  model  data. 

Although  commander  mental  model  congruence  occasionally  reached  60%  to  70%,  overall 
mental  model  congruence  was  more  frequently  around  50%  or  lower.  As  discussed  earlier,  a 
variety  of  potential  reasons  for  this  exist,  from  lack  of  shared  understanding  of  the  O&O  concept 
to  possible  need  for  improvement  in  the  battle  command  tools  (e.g.,  automated  battle  damage 
assessment;  warnings  when  sensor  coverage  is  lost  from  named  areas  of  interest;  improved 
collaborative  planning  software). 

According  to  the  commander’s  mental  model  across  echelons  (figure  8),  the  least  experienced 
echelon  (CAB  level)  usually  had  the  highest  congruence  of  the  threat  to  their  mission  at  least 
50%  of  the  time.  They  achieved  higher  and  more  congruent  levels  than  any  other  command 
level.  This  also  held  true  for  peer  mental  models  as  shown  in  figure  8.  In  figure  8,  no  data  were 
available  for  the  UE  commander  team  on  June  18  p.m.  or  the  CAB  commander  team  on  June  22 
p.rn.  because  the  commander  did  not  complete  the  mental  model. 
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Figure  8.  Commanders’  mental  models. 


Team  mental  models  are  displayed  in  figure  9.  In  figure  9,  no  data  were  available  for  the  CAB 
team  mental  model  on  June  22  pan.  because  not  enough  members  completed  the  team  model. 
Although  congruence  is  less  among  the  same  peer  group,  once  again,  the  lowest  echelon 
command  group  achieved  the  highest  and  most  consistent  mental  model  congruence.  There  are 
several  possible  explanations  that  may  be  driving  this  finding: 

The  CAB  has  the  smallest  area  of  influence  and  operations,  and  the  most  entities 
(sometimes  down  to  vehicle  level)  to  control.  As  such,  they  are  much  more  focused  on  a 
narrow,  less  complex  mission,  allowing  more  congruence  among  the  command  team. 

Also,  several  company  commanders  were  sitting  in  the  vicinity  of  the  CAB  commander, 
which  would  facilitate  coordination.  Finally,  one  SME  pointed  out  that  the  lower  level 
commanders  had  the  least  familiarity  with  the  scenario,  unlike  higher  level  commanders 
who  had  prior  experience  in  other  FCS  UE  staff-level  exercises  at  Fort  Leavenworth, 
Kansas.  This  lack  of  familiarity  could  result  in  the  CAB-level  commanders  being  more 
focused  on  everything  about  the  fight  in  their  area  of  interest,  which  in  turn  provided  better 
mental  model  congruence. 

As  all  command  levels  had  the  same  equipment  available  to  them  to  plan  and 
communicate,  the  CAB  was  using  different  techniques  to  plan  and  execute  the  battle, 
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which  were  more  efficient  at  relaying  or  transmitting  the  commander’s  mental  model.  One 
technique  of  interest  was  a  relatively  lengthy  daily  AAR.  Although  the  UA  also  had  an 
AAR,  it  was  more  restricted  to  the  staff  versus  subordinate  commanders,  and  of  less 
duration,  since  the  UA  commander  also  had  other  exercise  responsibilities. 


Figure  9.  Team  mental  models. 


4.  Discussion 


In  contrast  with  the  relatively  high  self-assessments  of  SU  and  effectiveness,  there  was  relatively 
low  shared  understanding  of  the  main  threats  to  mission  accomplishment  among  teams  of 
commanders.  This  suggests  that  the  ability  to  act  collaboratively  may  be  limited  despite  the 
networked  battle  command  technology.  SU  and  shared  understanding  take  place  in  the  heads  of 
commanders,  not  on  the  network.  Thus,  for  command  teams  to  be  successful,  commanders  must 
leam  to  periodically  share  their  understanding  of  the  tactical  situation  (Serfaty,  Entin,  & 
Johnston,  1998),  as  well  as  have  an  understanding  of  the  roles  and  responsibilities  of  other  team 
members  (Mathieu  et  ah,  2000). 
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It  is  interesting  to  note  that  the  lowest  level  of  command  had  the  highest  and  most  consistent 
levels  of  congruence.  This  could  be  attributable  to  a  simpler  mission  plan  or  to  the  fact  that  the 
team  was  using  techniques  not  employed  by  the  other  levels  of  the  command  team. 

Given  the  confounding  in  the  design  of  the  experiment,  it  is  impossible  to  determine  the  relative 
roles  played  by  doctrine  or  the  understanding  of  doctrine,  the  ad  hoc  nature  of  the  team,  the 
limited  training,  the  current  organization,  or  the  limitations  of  the  battle  command  technology 
itself. 

Mental  model  congruence  did  not  increase  substantially  over  the  experiment,  suggesting  that 
team  learning  did  not  occur.  Again,  whether  this  was  attributable  to  the  nature  of  the  team, 
organization,  training,  or  battle  command  technology  is  difficult  to  detennine. 

There  is  a  major  discrepancy  between  subjective  measures  of  workload  (low  and  generally 
decreased  over  the  experiment),  effectiveness,  and  SU  (both  were  high  and  if  anything, 
increased)  and  more  objective  measures  of  mental  model  congruence. 


5.  Conclusions 


There  are  two  conclusions  that  we  think  are  most  likely  to  be  drawn  from  this  analysis.  First, 
numerous  variables  (or  factors)  were  interacting  with  the  battle  command  staff  that  prevented 
them  from  establishing  and  maintaining  a  coherent  mental  model  of  the  battle  plan  and  resulting 
fight;  however,  these  variables  do  not  have  a  similar  detrimental  effect  on  workload  and  SU. 
Second,  it  appears  that  the  subjective  measurements  are  showing  more  subject  bias  to  questions 
on  the  survey. 

Controlled  experiments  are  needed  for  us  to  understand  the  relationship  of  doctrine,  organization, 
training,  and  battle  command  tools  and  to  isolate  the  influence  of  these  factors  and  their  effect  on 
the  battle  staff  in  maintaining  a  coherent  mental  model  of  the  battle  plan  and  resulting  fight. 
Having  a  dedicated  team  of  players  (e.g.,  those  currently  assigned  to  the  UA  experimental 
element)  should  help  reduce  the  role  played  by  variations  in  shared  understanding  of  doctrine, 
limited  training,  and  ad  hoc  team  composition. 

Smaller  scaled,  structured  experiments  with  a  dedicated  team  of  players  would  have  several 
advantages.  Smaller  scale  experiments  would  reduce  the  complexity  of  training,  since  there  are 
fewer  personnel  and  fewer  roles  to  train  plus  less  complexity  of  development  of  doctrine  and 
TTPs,  since  there  are  fewer  organizational  levels  involved.  Also,  these  experiments  would  enable 
better  data  on  workload,  SU  and  mental  models.  In  smaller  scaled,  structured  experiments,  it 
would  be  possible  to  have  objective  measures  of  workload  (e.g.,  performance  of  secondary 
tasks);  SU  (e.g.,  SAGAT,  as  mentioned  earlier);  and  mental  models  (e.g.,  having  team  members 
periodically  indicate  their  most  important  task,  biggest  problem,  and  resource  needed).  In  turn, 
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these  data  would  enable  researchers  to  determine  the  quality  of  mental  models,  workload,  and 
SA  and  how  these  concepts  are  affected  by  doctrine,  organization,  training,  and  materiel.  Such 
experiments  would  also  help  us  determine  if  mental  model  congruence  increases  over  time  and 
elucidate  relationships  between  mental  models,  workload,  and  SA  (e.g.,  whether  differences  in 
levels  of  these  variables  are  attributable  to  differences  in  measurement  techniques  or  to  more 
fundamental  causes). 
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